Impact of a machine learning algorithm on time to palliative care in a primary care population: protocol for a stepped-wedge pragmatic randomized trial.

BACKGROUND: As primary care populations age, timely identification of palliative care need is becoming increasingly relevant. Previous studies have targeted particular patient populations with life-limiting disease, but few have focused on patients in a primary care setting. Toward this end, we propose a stepped-wedge pragmatic randomized trial whereby a machine learning algorithm identifies patients empaneled to primary care units at Mayo Clinic (Rochester, Minnesota, United States) with high likelihood of palliative care need. METHODS: 42 care team units in 9 clusters were randomized to 7 wedges, each lasting 42 days. For care teams in treatment wedges, palliative care specialists review identified patients, making recommendations to primary care providers when appropriate. Care teams in control wedges receive palliative care under the standard of care. DISCUSSION: This pragmatic trial therefore integrates machine learning into clinical decision making, instead of simply reporting theoretical predictive performance. Such integration has the possibility to decrease time to palliative care, improving patient quality of life and symptom burden. TRIAL REGISTRATION: Clinicaltrials.gov NCT04604457 , restrospectively registered 10/26/2020. PROTOCOL: v0.5, dated 9/23/2020.

Effect of an Artificial Intelligence Decision Support Tool on Palliative Care Referral in Hospitalized Patients: A Randomized Clinical Trial.

CONTEXT: Palliative care services are commonly provided to hospitalized patients, but accurately predicting who needs them remains a challenge. OBJECTIVES: To assess the effectiveness on clinical outcomes of an artificial intelligence (AI)/machine learning (ML) decision support tool for predicting patient need for palliative care services in the hospital. METHODS: The study design was a pragmatic, cluster-randomized, stepped-wedge clinical trial in 12 nursing units at two hospitals over a 15-month period between August 19, 2019, and November 17, 2020. Eligible patients were randomly assigned to either a medical service consultation recommendation triggered by an AI/ML tool predicting the need for palliative care services or usual care. The primary outcome was palliative care consultation note. Secondary outcomes included: hospital readmissions, length of stay, transfer to intensive care and palliative care consultation note by unit. RESULTS: A total of 3183 patient hospitalizations were enrolled. Of eligible patients, A total of 2544 patients were randomized to the decision support tool (1212; 48%) and usual care (1332; 52%). Of these, 1717 patients (67%) were retained for analyses. Patients randomized to the intervention had a statistically significant higher incidence rate of palliative care consultation compared to the control group (IRR, 1.44 [95% CI, 1.11-1.92]). Exploratory evidence suggested that the decision support tool group reduced 60-day and 90-day hospital readmissions (OR, 0.75 [95% CI, 0.57, 0.97]) and (OR, 0.72 [95% CI, 0.55-0.93]) respectively. CONCLUSION: A decision support tool integrated into palliative care practice and leveraging AI/ML demonstrated an increased palliative care consultation rate among hospitalized patients and reductions in hospitalizations.

Improving time to palliative care review with predictive modeling in an inpatient adult population: study protocol for a stepped-wedge, pragmatic randomized controlled trial.

BACKGROUND: Palliative care is a medical specialty centered on improving the quality of life (QOL) of patients with complex or life-threatening illnesses. The need for palliative care is increasing and with that the rigorous testing of triage tools that can be used quickly and reliably to identify patients that may benefit from palliative care. METHODS: To that aim, we will conduct a two-armed stepped-wedge cluster randomized trial rolled out to two inpatient hospitals to evaluate whether a machine learning algorithm accurately identifies patients who may benefit from a comprehensive review by a palliative care specialist and decreases time to receiving a palliative care consult in hospital. This is a single-center study which will be conducted from August 2019 to November 2020 at Saint Mary's Hospital & Methodist Hospital both within Mayo Clinic Rochester in Minnesota. Clusters will be nursing units which will be chosen to be a mix of complex patients from Cardiology, Critical Care, and Oncology and had previously established relationships with palliative medicine. The stepped wedge design will have 12 units allocated to a design matrix of 5 treatment wedges. Each wedge will last 75 days resulting in a study period of 12 months of recruitment unless otherwise specified. Data will be analyzed with Bayesian hierarchical models with credible intervals denoting statistical significance. DISCUSSION: This intervention offers a pragmatic approach to delivering specialty palliative care to hospital patients in need using machine learning, thereby leading to high value care and improved outcomes. It is not enough for AI to be utilized by simply publishing research showing predictive performance; clinical trials demonstrating better outcomes are critically needed. Furthermore, the deployment of an AI algorithm is a complex process that requires multiple teams with varying skill sets. To evaluate a deployed AI, a pragmatic clinical trial can accommodate the difficulties of clinical practice while retaining scientific rigor. TRIAL REGISTRATION: ClinicalTrials.gov NCT03976297 . Registered on 6 June 2019, prior to trial start.

Improving the delivery of palliative care through predictive modeling and healthcare informatics.

OBJECTIVE: Access to palliative care (PC) is important for many patients with uncontrolled symptom burden from serious or complex illness. However, many patients who could benefit from PC do not receive it early enough or at all. We sought to address this problem by building a predictive model into a comprehensive clinical framework with the aims to (i) identify in-hospital patients likely to benefit from a PC consult, and (ii) intervene on such patients by contacting their care team. MATERIALS AND METHODS: Electronic health record data for 68 349 inpatient encounters in 2017 at a large hospital were used to train a model to predict the need for PC consult. This model was published as a web service, connected to institutional data pipelines, and consumed by a downstream display application monitored by the PC team. For those patients that the PC team deems appropriate, a team member then contacts the patient's corresponding care team. RESULTS: Training performance AUC based on a 20% holdout validation set was 0.90. The most influential variables were previous palliative care, hospital unit, Albumin, Troponin, and metastatic cancer. The model has been successfully integrated into the clinical workflow making real-time predictions on hundreds of patients per day. The model had an "in-production" AUC of 0.91. A clinical trial is currently underway to assess the effect on clinical outcomes. CONCLUSIONS: A machine learning model can effectively predict the need for an inpatient PC consult and has been successfully integrated into practice to refer new patients to PC.

Challenges of Modeling Outcomes for Surgical Infections: A Word of Caution.

Background: We developed a novel analytic tool for colorectal deep organ/space surgical site infections (C-OSI) prediction utilizing both institutional and extra-institutional American College of Surgeons-National Surgical Quality Improvement Program (ACS-NSQIP) data. Methods: Elective colorectal resections (2006-2014) were included. The primary end point was C-OSI rate. A Bayesian-Probit regression model with multiple imputation (BPMI) via Dirichlet process handled missing data. The baseline model for comparison was a multivariable logistic regression model (generalized linear model; GLM) with indicator parameters for missing data and stepwise variable selection. Out-of-sample performance was evaluated with receiver operating characteristic (ROC) analysis of 10-fold cross-validated samples. Results: Among 2,376 resections, C-OSI rate was 4.6% (n = 108). The BPMI model identified (n = 57; 56% sensitivity) of these patients, when set at a threshold leading to 80% specificity (approximately a 20% false alarm rate). The BPMI model produced an area under the curve (AUC) = 0.78 via 10-fold cross- validation demonstrating high predictive accuracy. In contrast, the traditional GLM approach produced an AUC = 0.71 and a corresponding sensitivity of 0.47 at 80% specificity, both of which were statstically significant differences. In addition, when the model was built utilizing extra-institutional data via inclusion of all (non-Mayo Clinic) patients in ACS-NSQIP, C-OSI prediction was less accurate with AUC = 0.74 and sensitivity of 0.47 (i.e., a 19% relative performance decrease) when applied to patients at our institution. Conclusions: Although the statistical methodology associated with the BPMI model provides advantages over conventional handling of missing data, the tool should be built with data specific to the individual institution to optimize performance.

An isogenic blood-brain barrier model comprising brain endothelial cells, astrocytes, and neurons derived from human induced pluripotent stem cells.

The blood-brain barrier (BBB) is critical in maintaining a physical and metabolic barrier between the blood and the brain. The BBB consists of brain microvascular endothelial cells (BMECs) that line the brain vasculature and combine with astrocytes, neurons and pericytes to form the neurovascular unit. We hypothesized that astrocytes and neurons generated from human-induced pluripotent stem cells (iPSCs) could induce BBB phenotypes in iPSC-derived BMECs, creating a robust multicellular human BBB model. To this end, iPSCs were used to form neural progenitor-like EZ-spheres, which were in turn differentiated to neurons and astrocytes, enabling facile neural cell generation. The iPSC-derived astrocytes and neurons induced barrier tightening in primary rat BMECs indicating their BBB inductive capacity. When co-cultured with human iPSC-derived BMECs, the iPSC-derived neurons and astrocytes significantly elevated trans-endothelial electrical resistance, reduced passive permeability, and improved tight junction continuity in the BMEC cell population, while p-glycoprotein efflux transporter activity was unchanged. A physiologically relevant neural cell mixture of one neuron: three astrocytes yielded optimal BMEC induction properties. Finally, an isogenic multicellular BBB model was successfully demonstrated employing BMECs, astrocytes, and neurons from the same donor iPSC source. It is anticipated that such an isogenic facsimile of the human BBB could have applications in furthering understanding the cellular interplay of the neurovascular unit in both healthy and diseased humans. Read the Editorial Highlight for this article on page 843.